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The aims of this study were; 1) to develop the final version of the Korean Roland-Morris 
Disability Questionnaire (RDQ), and 2) to compare the responsiveness between the RDQ 
and the Oswestry Disability Index (ODI) scores in patients having low back pain. The 
psychometric properties of the final Korean RDQ were evaluated in 221 patients. Among 
them, 30 patients were reliability tested. Validity was evaluated using an 11 -point 
numerical rating scale (NRS) and the Korean ODI. The receiver operating characteristic 
(ROC) curve analysis of the RDQ and the ODI was compared in 54 patients with lumbar 
zygapophyseal (facet) joint pain. There was a moderate relationship between the RDQ and 
NRS (r= 0.59, P< 0.01) and a strongly positive correlation between the RDQ and the ODI 
(r= 0.76, P< 0.001). The Korean RDQ with the higher area under the ROC curve showed 
a better overall responsive performance than did the ODI in patients with lumbar facet 
joint pain after medial branch radiofrequency neurotomy [P< 0.01). The results of the 
study present the final version of the Korean RDQ is valid for assessing functional status in 
a Korean population with chronic low back pain. 

Key Words: Low Back Pain; Oswestry Disability Index; Psychometric Properties; Roland- 
Morris Disability Questionnaire 



INTRODUCTION 

Low back pain (LBP) has been a major public health burden and 
a leading cause of work disability for many years. Among adults 
in the general population, 70%-85% will experience at least one 
episode of LBP at some time during their lives (1). Because LBP 
is not a clinical entity but a symptom with different stages of im- 
pairment, disability, and chronicity, physical examination and 
objective measures of function are at best only weakly associated 
with outcomes that are more relevant to patients, being symp- 
tom relief, daily functioning, and back pain-related disability (2, 
3). Assessment of these patient-oriented factors in routine clini- 
cal practice is, therefore, essential for clinical research and the 
assessment of the patients' progress (4). 

The Roland-Morris Disability Questionnaire (RDQ) and the 
Oswestry Disability Index (ODI) have been the 2 most common- 
ly used questionnaires to measure and monitor changes in func- 
tional outcome in patients with LBP (5). The RDQ evaluates 24 
activity limitations due to LBP, while the ODI consists of 10 items 
assessing the level of pain interference with physical activities. 



They are both easy to use and highly correlated, having similar 
ability to detect functional changes over time (6, 7). However, 
because of the floor effect of the ODI and the ceiling effect of 
the RDQ (5), the ODI is suggested to be the better choice in pop- 
ulations with higher disability levels, while the RDQ is recom- 
mended for general populations where a number of individuals 
are at the lower end of the disability spectrum (8, 9). 

Although there remain several issues regarding measurement 
of the psychometric properties understudied (10), it is impor- 
tant to validate the patient-oriented questionnaires in different 
languages which facilitate the exchange of information in inter- 
national studies. The ODI and the RDQ have been developed in 
several languages after comprehensive validity testing to verify 
what the questionnaire intends to measure. In a previous study, 
the 9-item Korean version of the ODI, in which questions in sec- 
tion 8 (concerning sex life) was omitted from the original ODI 
(version 2.0), was validated in patients with lumbar spinal dis- 
orders who had higher disability and underwent surgical oper- 
ations ( 1 1). As for the RDQ, a Korean linguistic validation version 
has been available since 2005 (www.rmdq.org/downloads/ko- 
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rean.pdf ), but the psychometric properties has not been exam- 
ined, yet. 

The aims of this study were; 1) to develop the final version of 
the Korean RDQ with evaluation of its psychometric properties, 
and 2) to compare the responsiveness between the RDQ and 
the ODI scores in patients with LBP who have a relatively mild 
to moderate disability. 

MATERIALS AND METHODS 

The study population consisted of 221 participants (73 men and 
148 women). The inclusion criteria were the following: adults 
aged between 18 and 80 yr, LBP (with or without radiation to leg) 
in chronic phase (pain lasting more than 3 months). The exclu- 
sion criteria were: patients with cognitive deficits discovered dur- 
ing the interview; patients with LBP thought to be caused by in- 
fectious disease, malignancy, and visceral diseases; patients with 
other pathologies causing disabilities or impairments affecting 
their daily tasks, such as lower-limb amputation or disorders 
affecting joint mobility. 

The assessments included age, gender, weight, height, dura- 
tion of LBP and the 1 1 -point numerical rating scale (NRS), where 
0 represented no LBP and 10 worst imaginable LBP. The 11 -point 
NRS was limited to their LBP. Functional disability was evaluat- 
ed using the Korean version of the RDQ and the 9-item Korean 
version of the ODI. In the RDQ, each positive response is scored 
+ 1 and each negative response (question without mark) is scored 
0, yielding a final score ranging from 0 (no disability) to 24 (max- 
imum disability). The RDQ did not present unanswered ques- 
tions or missed information, however, the Korean ODI with 1 
question without an answer was corrected as total score/(5 x 
number of questions answered) x 100%. For assessing respon- 
siveness, the seven-point global perceived effect (GPE) scale was 
applied for the patients who had been diagnosed with lumbar 
facet joint pain and had undergone lumbar medial branch ra- 
diofrequency (MBRF). 

The study was divided in 3 phases. 

Phase I. Translation and cross-cultural adaptation 

The procedure included 2 forward translations, comparison 
among 2 translations and a Korean linguistic validation version 
previously performed by MAPI Research (www.rmdq.org/down- 
loads/korean.pdf ), synthesis, 2 back translations, expert com- 
mittee examination to produce a pre-final version, testing of the 
pre-final version, and the development of the final version. 

Initially, the two native speakers in Korean language without 
knowledge of the Korean linguistic validation of the RDQ, per- 
formed forward translation of the RDQ from English into Kore- 
an independenuy; one of them was an anesthesiologist familiar 
with the concept of the questionnaires and the other was a pro- 
fessional translator (an English teacher) who was uninformed 



of the research. 

Then, the expert committee, formed by two translators, two 
pain clinicians, and a methodologist reviewed the 2 translations 
and compared them with a Korean linguistic validation version 
of the RDQ by MAPI Research. After careful review, some mod- 
ifications in the 7 sentences of the Korean linguistic validation 
version were decided according to both linguistic and cultural 
adaptation because 2 newly translated versions were basically 
similar with a previous Korean linguistic validation version ex- 
cept for the 7 questions. The next step was backward translations 
performed independenuy by 2 back translators who were bilin- 
gual in Korean and English, but English mother tongues; one of 
them was a Korean-American physical therapist and the other 
was a Canadian professional translator. They never had knowl- 
edge of the original English version of the RDQ. 

Then, the expert committee, in whom 2 backward translators 
were newly included, reviewed and discussed the original ques- 
tionnaire and the 2 versions of the translation and came to agree- 
ment on the semantic, idiomatic, experiential, and conceptual 
equivalence between the source and the target version (the pre- 
final version). 

The pre-final version of the RDQ was handed out to 33 pa- 
tients who, as classified by pain clinicians, were suffering from 
chronic LBP without much change in symptoms. They were re- 
administered the questionnaire 1 week later during an outpa- 
tient visit (10). They were also requested to stop all the interven- 
tional procedures newly performed for their low back complaints 
until the next visit except for their oral medications and ongo- 
ing physical therapy. Three patients were not re-administered 
the follow-up after their first response; two of them visited our 
clinic more than 3 weeks later from their first response and one 
underwent an acupunctural procedure for his aggravating low 
back pain. Patients were also asked to highlight those sentences 
in each category of the pre-final Korean RDQ which were un- 
clear or those which gave them uncertainty when their first ad- 
ministration of the pre-final Korean RDQ. All the findings from 
this step were re-evaluated by the expert committee, although 
no adjustment was further required, and this version was con- 
sidered the final Korean RDQ version. Therefore, the first data 
from 30 patients in phase I was included in statistical analysis of 
Phase II, psychometric properties examination of the final ver- 
sion of the Korean RDQ. 

Phase II. Psychometric properties examination of the final 
version 

The final Korean RDQ version was applied to 200 new patients. 
The inclusion criteria and recorded variables were the same as 
the aforementioned ones. Nine patients did not sign the consent 
form or refused to participate in this study for personal reasons, 
therefore, a total of 221 patients, comprised of 30 patients in phase 
I and 191 patients in phase II, were finally included for psycho- 
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metric properties examination of the final Korean RDQ version. 
However, the results of re-tests in 30 patients from Phase I was 
not included in this phase to avoid carry-over effect of patient 
responses on the measure. 

Phase III. Responsiveness of the final version 

For the assessment of responsiveness, we selected 58 patients 
among the 191 patients in Phase II, who were diagnosed (by 
dual controlled comparative local anesthetic blocks) with lum- 
bar facet joint pain and scheduled for lumbar MBRF. There was 
no patient from Phase I in Phase III. Before undergoing lumbar 
MBRF, patients completed baseline NRS, ODI and RDQ which 
were gathered in Phase II. After lumbar MBRF by using the tun- 
nel vision technique (12), follow-up questionnaires and the sev- 
en-point GPE scale were obtained from 54 patients at the 4-week 
post-MBRF follow-up meeting. In the seven-point GPE scale, 
patients were asked to score the change in their LBP-caused func- 
tional limitations compared to their situations before having un- 
dergone lumbar MBRF (Phase II). The scale criteria were: 1 = 
worse than ever, 2 = much worsened, 3 = slightly worsened, 4 = 
unchanged, 5 = slightly improved, 6 = much improved, 7 = com- 
pletely recovered. 

Analysis 

The psychometric properties of the final Korean RDQ version 
were analyzed using SPSS version 17.0 (SPSS, Chicago, IL, USA). 
A list of the tests follows: 

The test-retest reliability (repeatability) was evaluated using 
the Intraclass Correlation Coefficient (ICC). ICC value above 0.70 
is considered acceptable (13). We also used the constructed Bland 
Airman Plot (14) to visualize the dispersion of the measurements 
and; the mean of the 2 scores on the RDQ determined at base- 
line and after 1 week The differences between these scores were 
plotted against each other for each subject. Internal consistency 
reliability (homogeneity) was estimated by means of Cronbach's 
alpha (15). 

The convergent validity was investigated by comparing the 
response to the RDQ with result of an 11 -point NRS at baseline 
and by correlating the final Korean RDQ and the Korean ODI. 
For all correlations, Pearson's correlation coefficients were used 
because the variables were either parametric or normally dis- 
tributed. A strong correlation was considered to be over 0.60, a 
moderate between 0.30 and 0.60, and a low (very low) correla- 
tion below 0.30 (13). 

Potential ceiling and floor effects were measured by assess- 
ing the distribution of answers in the categories and calculating 
the percentage of patients indicating the minimum and maxi- 
mum possible scores in both questionnaires. Ceiling and floor 
effects are considered to be present if more than 15% of respon- 
dents achieved the lowest or highest possible total score (13). 

To assess responsiveness we used the 7-point GPE scale which 



was recommended as the most common external criterion (16), 
to classify the patients into groups: Those with a score of 6-7 
points as "improved" 3-4-5 as "unchanged" and 1-2 as the "de- 
teriorated" group (10). Because the definition of clinical improve- 
ment requires a dichotomous variable, we compared the "im- 
proved" and the "unchanged" groups by calculating the chang- 
es in RDQ before and after lumbar MBRF. The receiver operat- 
ing characteristic (ROC) curve analysis was performed to deter- 
mine the optimal cut-off point of RDQ change for improvement. 
We calculated the area under the curve (AUC) which reflects the 
ability of the test to discriminate between these two categories 
of subjects and ranges from 0.5 (no accuracy in distinguishing 
improved from unchanged) to 1.0 (perfect accuracy). A change 
in score larger than the optimal cut-off point is taken to be an 
important change. In addition, by using Z test we compared the 
2 ROC curves (ODI change vs. RDQ change) to explore the abil- 
ity of the questionnaires to detect change with higher diagnos- 
tic validity in patients with lumbar facet joint pain. 

Ethics statement 

The study was approved by the institutional review board of Seoul 
National University Bundang Hospital (IRB No. B-0912-090-007) 
and carried out on patients with LBP referred to the outpatient 
clinic of our institution between November 2009 and April 2010. 
All patients received written and oral information about the study 
and gave their written consent to participate. 

RESULTS 

The populations studied in Phase I and II were similar with re- 
gard to the parameters measured, therefore, the data of both 
phases are presented together. Table 1 shows the clinical and 
demographic characteristics of the study participants. 

Phase I. Translation and cross-cultural adaptation 

Some modifications of the 7 sentences in the Korean linguistic 
validation version of the RDQ were performed by the expert com- 
mittee after careful reviews with 2 different forward translations; 
4 resulted from awkward expressions, 2 from the differences in 
nuisances, and 1 due to cultural difference. For example, in ques- 



Table 1. Demographic and clinical characteristics of study population (n = 221) 



Parameters 


Mean 


SD 


Maximum 


Minimum 


Median 


Age (yr) 


55.8 


15.4 


21 


80 


57 


Height (cm) 


161.5 


9.3 


140.0 


187.0 


160.5 


Weight (kg) 


63.5 


10.5 


40.4 


102.3 


63.0 


Pain duration (mo) 


18.6 


18.5 


3 


120 


12 


NRS (0-10) 


5.8 


2.1 


10 


0 


6 


RDQ (0-24) 


8.6 


5.3 


24 


0 


8 


ODI (0-100) 


37.3 


16.2 


91 


0 


36 



SD, standard deviation; NRS, numerical rating scale; RDQ, Roland-Morris Disability 
Questionnaire; ODI, Oswestry Disability Index. 
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tion No. 7, an easy chair is only used by a few percent of Korean 
people, the word "easy" was omitted from the statement "Be- 
cause of my back, I have to hold on to something to get out of 
an easy chair" Because of the differences in Korean expressive 
and honorific language, "I am not doing any of the jobs that I 
usually do around the house" was changed to "I cannot do any 
of the jobs that I usually do around the house" in question No. 4. 
In addition, "I try to get other people to do things for me" was 
clarified to "I try to ask other people to do things for me" in ques- 
tion No 8. There were two severely awkward sentences in the Ko- 
rean linguistic validation version of the RDQ by MAPI research. 
Question No. 13 in Korean linguistic validation version of the 
RDQ by MAPI research was an awkwardly translated sentence, 
therefore changed into a usual expression in the Korean lan- 
guage. Question No. 18 in the Korean linguistic validation ver- 
sion of the RDQ by MAPI research was wrong, because the orig- 
inal translation confuses time of sleep with quality of sleep and 
therefore was changed. Finally, there were 2 minor revisions in 
question No. 9 and 12 in their nuisances. 

Phase II. Psychometric properties examination of the final 
version 

The test-retest reliability (repeatability) was satisfactory. The ICC, 
estimated by considering 30 patients on two occasions separated 
by a mean time interval of 7.76 days (range 4-9 days) was 0.989 
{P < 0.001). The constructed Bland and Altaian plot for test-re- 
test agreement showed a good reliability (Fig. 1). The internal 
consistency by means of Cronbach's alpha was 0.879 with coef- 
ficients ranging from 0.865 (question No. 4) to 0.886 (question 
No. 2). 
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Fig. 1 . Bland Altman plot for test-retest reliability of the Roland-Morris Disability Ques- 
tionnaire: mean-against-difference graph. For each subject (n = 30), the mean of the 
2 scores on the Korean RDQ at baseline (TO) and after 4-9 days (11) and the differ- 
ence between these time points (T1 -TO) are plotted against each other. 



The correlation between the RDQ and the 1 1 -point NRS show- 
ed a positive and statistically significant association (r = 0.59, 
P < 0.001). The assessment of the convergent validity showed a 
strongly positive correlation between the RDQ and the ODI (r = 
0.76, P < 0.001). 

The ceiling and floor analysis was performed on the total score 
of the RDQ. The percentage of respondents who achieved the 
lowest score (floor effect) was 0.4% (1 patient) and the highest 
score (ceiling effect) was 0.9% (2 patients). There were no floor 
and ceiling effects for the RDQ total scores. 

Phase III. Responsiveness of the Final Version 

The responsiveness of the RDQ was examined by comparing 
the baseline data and the data collected 1 month after lumbar 
MBRF in 54 patients (23 men and 31 women) with chronic lum- 
bar facet joint pain. Table 2 presents the characteristics of pa- 
tients in Phase III. According to the 7-point GPE scale, 32 patients 
(59.3%) reported to have improved (6-7 points in the 7-point 
GPE scale), 20 patients (37.0%) to be unchanged (3-5 points in 
the 7-point GPE scale), and 2 patients (3.7%) to be deteriorated 



Table 2. Characteristics of patients in Phase III (n = 54) 



Variables 


Findings 


Age (yr, SD) 


58.0(11.2) 


Sex (male/female) 


23/31 


Height (cm, SD) 


162.6(9.8) 


Weight (kg, SD) 


64.9(10.2) 


Pain duration (mo, SD) 


16.0(12.9) 


Baseline score of NRS (SD) 


6.1 (1.9) 


Baseline score of RDQ (SD) 


9.8(4.1) 


Baseline score of ODI (%, SD) 


38.8(13.5) 



SD, standard deviation; NRS, numerical rating scale; RDQ, Roland-Morris Disability 
Questionnaire; ODI, Oswestry Disability Index. 




0 20 40 60 80 100 

100-Specificity 

Fig. 2. Area under the receiver operating characteristic curves for the Roland-Morris 
Disability Questionnaire and Oswestry Disability Index at one month. 
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(1-2 points in the 7-point GPE scale). As forementioned, the "im- 
proved" and "unchanged" groups were included for analysis. 
The area under the ROC curve of the RDQ changed after lum- 
bar MBRF was 0.958 (95% confidence interval, 0.86-0.99). The 
optimal cut-off value was 3.5 with 95.0% in sensitivity and 81.2% 
in specificity. The RDQ produced a significantly higher area un- 
der the receiver operating characteristic curve statistics (AUC = 
0.95; 95% CI, 0.85-0.93) than the ODI (AUC = 0.70; 95% CI, 0.56- 
0.82) in Fig. 2 (P = 0.001). 

DISCUSSION 

Our study showed that the Korean RDQ is reliable and valid in 
a Korean population with chronic LBP. The test-retest reliability 
of the Korean version of the RDQ was excellent and the internal 
consistency, assessed by means of Cronbach's alpha was similar 
to the values present in other studies (17-19). There was a posi- 
tive and moderate significant correlation between the RDQ and 
the NRS, coinciding with previous studies (17-19). The conver- 
gent validity assessed by means of the correlation between the 
Korean RDQ and the Korean ODI was 0.758, very similar to the 
one reported in other studies (17, 19). For responsiveness, in 
patients with chronic lumbar facet joint pain who underwent 
lumbar MBRF, the Korean RDQ with the higher area under the 
ROC curve showed a better overall responsive performance than 
did the Korean ODI. In those patients, based on the anchor based 
method using the 7-point GPE scale, the minimal important 
change of the RDQ was 3.5 points. 

For the clinical characteristics, the majority of the participants 
were female (male:female = 73:148) and they were not selected 
in this study but allowed in a consecutive way. This female pre- 
dominance can be explained by two different reasons. One is a 
general expectation that women are usually considered by soci- 
ety to be responsible for household activities and child rearing, 
even if the situation is changing and many women are now work- 
ing in Korea. The other is a special seasonal cause in Korea. We 
performed this study from November to the next April when the 
kimchi-making (gjmjang) days during the early December and 
Lunar New Year's holidays in the middle of February was includ- 
ed. Gimjang, which is an annual event especially for Korean 
women to prepare kimchi having for their family next year, re- 
quires heavy labor for women to lift heavy items with bending 
their spine and knees. Lunar New Year's holiday, which is one 
of the biggest festive seasons for many Koreans where ancestral 
rites are performed during family gathering, can be stressful 
events for women, creating a newly coined term like "holiday 
syndromes." In these seasons many Korean women suffering 
from LBP visit their doctors. A few studies also report a high prev- 
alence of LBP in women (20, 21). 

If the questionnaire has been well adapted and validated ac- 
cording to linguistic expression and cultural adaptation, com- 



parison of different research findings worldwide can also be pos- 
sible. Even though there are many scales to measure lumbar dis- 
ability in Korea, there is no culturally adapted and validated scale 
available to be used in this population except for the Korean ODI. 
Previous studies which performed clinical trials in a Korean pop- 
ulation were able to use very limited scales for evaluating LBP, 
such as visual analogue scale (VAS) or NRS (22, 23). We reviewed 
a previously existing version of the Korean linguistic validation 
RDQ and culturally corrected some awkward expressions in it. In 
particular, question 18 "I sleep less well on my back" was wrong- 
ly translated as "I sleep less because of my back pain" in the Ko- 
rean linguistic validation RDQ version, therefore we corrected 
and clarified it from time of sleep to quality of sleep. The final ver- 
sion of the Korean RDQ is easy to read and makes it appropriate 
for following-up the progress of individual patients in busy out- 
patient settings. It is hoped that this culturally equivalent out- 
come measurements will allow researchers to perform LBP stud- 
ies reliably in culturally diverse countries. 

The use of RDQ is commonly recommended for patients who 
Onts with persistent severe disability (5). Although a large num- 
ber of studies had reported on the responsiveness of the ODI and 
RDQ compared in a number of different populations (24-26), 
there was a paucity of research reporting the responsiveness of 
these measures for patients with mild to moderate LBP, focus- 
ing on a single diagnostic entity. This is the first study which com- 
pares both scores prospectively in a cohort of patients with chron- 
ic lumbar facet joint pain. Our study shows that the RDQ could 
be a suitable evaluation tool for patients with lumbar facet joint 
pain. In this study, patients with facet joint pain were chosen 
because it is assumed to be a relatively mild to moderate disabil- 
ity and of interest to many pain clinicians. However, the gener- 
alizability is needed in order to evaluate a responsiveness of this 
measurement tool. 

There are three limitations worth noting in this study. One 
limitation is the potential for a carry-over effect given that the 
two administrations for the test-retest reliability were close to- 
gether in time (mean time interval = 7.76 days), therefore indi- 
viduals simply remembering answers from the first administra- 
tion. Although the RDQ is widely recommended for assessing 
functional status in patients with low back pain and a previous 
study investigated its measurement properties including the 
time interval which influence RDQ agreement parameters (10), 
there is no definitive answer to determining the time interval; 
longer intervals have limitations as well. The second limitation 
is the strength of the relationship, particularly between the NRS 
and ODI which appeared to be a moderate relationship (r = 0.59). 
Because the NRS and RDQ measure different aspect, function 
and pain, they are not necessarily the same convergent validity 
being measured. Finally, a small sample size in phase III (n = 54) 
for evaluating responsiveness in the final version of the Korean 
RDQ is the third limitation. Baseline RDQ scores and the way to 
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cluster patients with regard to baseline scores are suggested to 
influence mainly the optimal cut-off point and responsiveness 
parameters (10). We had tried to divide two groups according to 
the median baseline RDQ score (median baseline RDQ score = 
9), however, sample sizes in each groups were too small to show 
generalization and this study was just designed to show the re- 
sponsiveness of RDQ in a homogenous group with a single dis- 
ease entity. Therefore, responsiveness should be tested in a ran- 
domized clinical trial design or a further follow-up study. In ad- 
dition, although there is a study which demonstrated that RDQ 
responsiveness was not influenced by type of intervention (10), 
further study about the responsiveness would be advantageous. 

In conclusion, the results of this study presents that the final 
version of the Korean RDQ is a reliable and valid tool for assess- 
ing the functional status in the Korean population with chronic 
LBR It is suggested that the Korean RDQ would be adequate for 
follow-up assessments of treatments in a busy clinical practice 
and can be recommended for use in clinical studies of quality 
of life in patients with LBP. 
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that the Korean RDQ would be adequate for follow-up assessments of treatments in a busy clinical practice and for clinical studies 
of quality of life in patients with low back pain. 
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